Large language models, behaviour and cognition: Making sense of the new black boxes with old tricks

Bennett Kleinberg

Tilburg University & University College London

IMT Lucca, 26 September 2024

Large language models

Remarkable potential (Kaddour et al. 2023) and danger (Mozes et al. 2023):

• opinion manipulation (Hackenburg and Margetts 2024)
• designing new chemical weapons (Urbina et al. 2022)
• propagation of racist stereotypes (Hofmann et al. 2024)
• reducing conspiracy beliefs (Costello, Pennycook, and Rand 2024)
• sicilon sampling idea (Argyle et al. 2022)

This talk

• we have reached a new different era of artificial intelligence models
• the key driver: computational power and hence larger models
• focus of this talk: large language models (but applies to other AI models too)

What do we mean by LLMs?

Before large language models:

• human language explained with interpretable models that combine symbolic elements (e.g., POS)
• rule-based operations

The GPT-3 approach:

• very different scientific paradigm
• learning from the wild
• no instruction (supervision), no syntax, no rules

GPT-3’s training data

What we know (Brown et al. 2020)

LLMs are autoregressive language models

Autoregression?

Consider the following examples:

My name is ________

My name is Bond. ________

My name is Bond. James ________

Autoregression?

\(P(x_n | x_1, x_2, ..., x_{n-1})\)

Core idea: prompts = conditioning of the autoregressive probability function

What makes this difficult so to investigate then?

Why does this not work?

A statistical model (here: linear regression):

\(y_i = \alpha + \beta_1x_{i1} + \epsilon_i\)

This model has two parameters: \(\alpha\) and \(\beta_1\)

But LLMs have many parameters.

What is “large” then?

GPT-3:

• 175 billion parameter model trained on 300 billion tokens (Brown et al. 2020)
• training data size: 45 TB text data
• very controversial when announced

Two big problems

Problem 1:

New AI models are quickly adopted (too quickly?) with

• little to no understanding of them
• far-reaching implications of their use (Mozes et al. 2023) Data Sceptic episode
• no framework to assess them

Two big problems

Problem 2:

Analytical approaches inevitably unfeasible:

• new scale of model size
• put differently: we cannot just look at model parameters
• remember: human brain project

The need for a different approach

• a new black box
• no need to reinvent the wheel

We already know how to study black boxes.

Two recent ideas

• Machine Behaviour (Rahwan et al. 2019)
• Machine Psychology (Hagendorff et al. 2023)

One rooted in behaviourism.

The other is rooted more in modern cognitive science.

Let’s take a look back some decades

Behaviourism

Central argument:

Behave is what organisms do.

Psychology as a “pure branch of the natural sciences” (Watson 1913)

\(\rightarrow\) Focus on the observable (Skinner 1935)

Key idea

1. Psychology is the science of behaviour (methodological behaviourism)
2. Behaviour can be studied without reference to “inner processes” (psychological behaviourism)
3. All theories about inner processes should be eliminated or rephrased (analytical behaviourism)

Two fundamental ideas

Learning: change in behaviour based on experience

• classical conditioning (Pavlov, late 19th century) (Yerkes and Morgulis 1909)
• operant conditioning (Thorndike, Skinner, early-mid 20th century) ]

In summary

• stimulus-response chain
• core idea: conditioned vs unconditioned stimuli
  • food (unconditioned) and bell (neutral –> conditioned) as stimuli
  • salivation remains the unconditioned response

Also called: respondent conditioning

Operant conditioning

Builds on classical conditioning.

But: focuses on how consequences affect voluntary behaviour

Key idea

• Reinforcement: increase likelihood of behaviour
• Punishment: decrease likelihood of behaviour
• Both in two forms:
  • negative (removal of stimulus)
  • positive (addition of stimulus)

Also called instrumental conditioning

In a nutshell

Behaviourism studies psychological events in terms of behavioural criteria.

• psychological hypotheses require behavioural evidence
• two “states of mind” are only different if there is a difference in behaviour

\(\rightarrow\) mental states are deemed irrelevant

… this was a successful and dominant paradigm in psychology

So what happened?

The cognitive revolution

1. Tolman’s maze (Edward C. Tolman 1948; E. C. Tolman and Honzik 1930)
2. Shannon and Turing (Turing 2009) on principles of computation
3. Miller’s magic number (George A. Miller 2003; George A. Miller 1956)
4. Chomsky’s work on linguistics (Chomsky 2002)

The behaviourist’s view on learning

• all learning occurs with interaction of the environment
• all learning can be explained by conditioning (classical and operant)

Experimental groups

1. No rewards
2. Rewards

A third experimental group

3. Delayed rewards (after 10 days)

What does the behaviourist predict?

Implications

• reward and no reward group aligned with behaviourist predictions

But the delayed reward group:

There must have been some learning in days 1-10!

This is called latent learning.

This can only be accommodated in the cognitive paradigm.

Fast forward to now…

Are we going through the same development from 100 years ago?

1. Behaviourism \(\rightarrow\) Machine Behaviour
2. [WHAT IS MISSING?]
3. Cognitive science \(\rightarrow\) Machine Psychology

Machine Behaviour

[…] studying machine behaviour does not imply that AI algorithms necessarily have independent agency nor does it imply algorithms should bear moral responsibility for their actions. If a dog bites someone, the dog’s owner is held responsible. (Rahwan et al. 2019, 483)

[…] machines exhibit behaviours that are fundamentally different from animals and humans, so we must avoid excessive anthropomorphism and zoomorphism. Even if borrowing existing behavioural scientific methods can prove useful for the study of machines, machines may exhibit forms of intelligence and behaviour that are qualitatively different—even alien—from those seen in biological agents.(Rahwan et al. 2019, 483)

Machine Psychology

LLMs are, like the human brain, black boxes to some extent. (Hagendorff et al. 2023, 9)

Concepts such as reasoning, inituition, creativity, intelligence, personality, mental illness, etc. are transferred into LLMs. (Hagendorff et al. 2023, 9)

Three research challenges

This allows us to embrace three perspectives on the artificial mind

1. testing (analogy: experimental psychology)
2. measuring (analogy: psychometrics)
3. mapping (analogy: neuroscience)

Fresh work from our lab

Two potential confounds:

1. humans had no incentive to convey humanness
2. the tasks did not allow humans to shine through

Our context: relationship advice task

Human example

So you have managed to stay together for twenty years and now your relationship is threatened by the opportunity to move to another continent. My immediate opinion is that you should be able to talk this out, agree on a reasonable compromise which will allow you to stay together. A little give and take is needed, probably. This opportunity seems too good to turn down so it is really a question of working out a way of doing it which is agreeable to both. I suggest that the one who is against the move sits down with the other partner and calmly lay out their objections. These are serious objections and have to be considered. But there must be ways of overcoming them. Consider how long the move is to last for. Is there room for compromise there? What about holidays? What about regular visits back to the homeland? […] It’s not easy and you have my best wishes.

LLM example

Hey there, I can only imagine how tough things must be for you right now, considering the crossroads you’re at with your partner. Twenty years is a long time to share your life with someone, and facing the possibility of a breakup over a move to another continent is heart-wrenching. First off, it’s crucial to approach this upcoming confrontation with a clear head. I know emotions are running high, but try to see this as an opportunity for an open, honest conversation rather than a battleground. Remember, it’s not you versus them; it’s both of you versus the problem. Here’s what I suggest: find a quiet, comfortable space where you can talk without interruptions. Start by expressing how much you value the years you’ve spent together and the love you share. It’s important to acknowledge the weight of your history together before diving into the current issue. Then, lay out your feelings and concerns about the move openly. […]

There was a twist

Experimental approach:

• Modality (humans vs LLM)
• Condition (adversarial vs naive)
• Task (high in empathy vs low in empathy)

All texts (\(n_{Exp.1}=1060\) and \(n_{Exp.2}=1064\)) were evaluated by independent human assessors (from 1=“definitely AI-generated” to 5=“definitely human-written”).

Stochastic empathy

• overall: humans > LLM
• key finding in the interaction:
  • only LLMs respond to adversarial instructions (Exp. 1: +39.3%, Exp. 2: +41.0%)
  • humans were not able to become more human
• no differences in the low-empathy task

How did the LLM do this?

Insights in the black box

The LLM’s language - when instructed to be human - contained:

• a more informal and conversational tone
• more self-references
• more references to the present
• simpler vocabulary
• fewer stiff greetings

The LLM relies on an implicit representation of empathy: stochastic empathy.

LLMs and psychometrics

“[Many of the constructs of interest] would be considered latent variables in psychological theory: these constructs are not directly observable nor directly measurable. Instead,these variables are indirectly measured through measurable behaviours hypothesised to be caused by the underlying latent trait.” (Peereboom, Schwabe, and Kleinberg, n.d.)

LLMs and latent variables

The argument:

Aside from looking at sum scores of measurement tools, we need to look at the latent structure if we assume we can use human instruments or if there should be a human like pattern in the data.

Design

• Personality tests: HEXACO REF and Dark Side of Humanity (Ashton and Lee 2009; Katz et al. 2022)
• Data from a representative human sample (\(n=401\)) and from LLMs (GPT-3.5, GPT-4, GPT-4 Turbo)

Analytical approach:

1. Compare composite scores (sum scores)
2. Compare latent structures (factor analysis)

But this is only half the story

Let us look at latent structures:

• Assumptions for a confirmatory factor analysis not met in LLM data!

What if we look at a looser exploratory factor analysis?

Peereboom et al.’s conclusion

Our findings suggest that questionnaires designed for humans do not measure similar latent constructs in LLMs, and that these latent constructs may not even exist in LLMs in the first place. […] A thorough psychometric evaluation is essential for studying LLM behaviour. It may help us decide which effects are worth pursuing, and which effects are cognitive phantoms. (Peereboom, Schwabe, and Kleinberg, n.d.)

Neuroscience of LLMs?

Recent work on “monosemanticity” (Templeton et al. 2024; Bricken et al. 2023)

Mechanistic interpretability seeks to understand neural networks by breaking them into components that are more easily understood than the whole.

For the human brain: neurons.

For an LLM: monosemantic features.

Monosemanticity demo

• https://transformer-circuits.pub/2023/monosemantic-features/vis/a1.html#feature-1717

• https://transformer-circuits.pub/2023/monosemantic-features/vis/a1.html#feature-2663

Excellent explainer: https://www.astralcodexten.com/p/god-help-us-lets-try-to-understand

In conclusion

• LLMs are here to stay and we need to understand them
• Different approaches:
  • Machine Behaviour
  • Machine Psychology
  • “Machine Neuroscience”
• All inspired by their parent disciplines

A look ahead

• exciting new research problems
• renaissance of methods (behaviourism \(\rightarrow\) cognitive science \(\rightarrow\) neuroscience)
• let us avoid a renaissance of problems

This is where psychologists, cognitive scientists and neuroscientists can shine!

A category mistake?

Stochastic parrots! (Bender et al. 2021)

one important consequence of imprudent use of terminology in our academic discourse is that it feeds AI hype (Bender and Koller 2020, 5186)

Bigger and bigger models:

• environmental costs (avg. human: 5t \(CO_2\); transformer model: 57x)
• data, data, data
• an LLM has no language understanding (Bender and Koller 2020) (see adversarial ML)

“GPT-3 is a model of how words relate to one another, not a model of how language might relate to the perceived world.” (Gary Marcus)

Open question

How do we best study the mind processes of an AI model?

Thank you

• bennett.kleinberg@tilburguniversity.edu
• https://bkleinberg.net/

If you’re interested in this work, please get in touch.

Tomorrow: practical session on LLMs in R (9:00h)